TESTING RATIONALITY OF COHERENT 
COHOMOLOGY OF SHIMURA VARIETIES 



MICHAEL HARRIS 
in memory of Ilya Piatetski- Shapiro 

Abstract. Let G" C G be an inclusion of reductive groups whose 
real points have a non-trivial discrete series. Combining ergodic 
methods of Burger-Sarnak and the author with a positivity argu- 
ment due to Li and the classification of minimal X-types of discrete 
series, due to Salamanca-Riba, we show that, if tt is a cuspidal au- 
tomorphic representation of G whose archimedean component is 
a sufficiently general discrete series, then there is a cuspidal au- 
tomorphic representation of G' , of (explicitly determined) discrete 
series type at infinity, that pairs non-trivially with tt. When G and 
G' are inner forms of U (n) and U{n — 1), respectively, this result is 
used to define rationality criteria for sufficiently general coherent 
cohomological forms on G. 



Introduction 

For years I would meet Ilya Piatetski-Shapiro at conferences and 
mathematical institutes, but we only spoke a few times, and our conver- 
sations were generally brief and to the point. Shortly after I arrived at 
the IAS in 1983 for the special year on automorphic forms - I later came 
to suspect that Ilya was in part responsible for my having been invited 
- he asked me to come to his office to tell him what I had been doing. 
I explained that I had been working on defining arithmetic models of 
automorphic vector bundles, to provide a general geometric framework 
for Shimura's characterization of arithmetic holomorphic automorphic 
forms - forms rational over number fields. Ilya asked whether I had a 
similar characterization of arithmetic non-holomorphic coherent coho- 
mology classes. I replied that I had not thought about the question; 
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he suggested that I work on the problem; and that was essentially the 
end of the discussionj^ 

Non-holomorphic coherent cohomology of Shimura varieties is usu- 
ally defined by automorphic representations whose archimedean com- 
ponents are in the non-holomorphic discrete series. Since Takeyuki 
Oda had also advised me to look at such automorphic representations, 
I spent the next few years developing a theory that provides a geo- 
metric framework for such cohomology classes, and in many cases I 
was able to find a characterization of the classes that are rational over 
number fields. But this characterization, developed in |H90j . is based 
on studying the cup products of the classes in question with holomor- 
phic arithmetic forms on Shimura subvarieties. These cup products are 
compatible with the rational structure of canonical models of Shimura 
varieties, and therefore the rationality of these cup products is a neces- 
sary condition for arithmeticity of the class being tested. The method 
of |H90] shows that it is often also a sufficient condition, but provides 
no information in the typical case, where all such cup products vanish 
for purely local reasons. 

The present paper gives an optimal answer to Piatetski-Shapiro's 
question for coherent cohomology of Shimura varietes attached to uni- 
tary groups, at least when the infinitesimal character is sufficiently 
regular. As in |H90] , the rationality criterion is based on restriction to 
Shimura subvarieties and integrating against classes on the latter, but 
this period integral in general is purely analytic and has nothing to do 
with algebraic geometry. Nevertheless, the multiplicity one condition 
implied by the Gross-Prasad conjecture |GGPj shows the existence of 
canonical period invariants, called the Gross-Prasad periods in |H12j . 
that have the property that the proportionality of the period integral 
to the corresponding Gross-Prasad period is a necessary and sufficient 
condition for arithmeticity. When the infinitesimal character is suffi- 
ciently regular, it can be shown that the period integrals do not all 
vanish. Thus the criterion provides a foolproof way to reduce arith- 
meticity on a Shimura variety attached to an inner form of U{n) to 
that on an inner form of U{n — 1), and by induction we obtain a gen- 
eral criterion. 

The remarks of the preceding paragraph need to be qualified, how- 
ever. The Gross-Prasad periods are defined in |H12j , assuming a stronger 
version of the local Gross-Prasad conjecture at archimedean places 



^He did ask one more question: was I married? I was not, and I was surprised 
to discover that he had advice for me on that matter as well. 
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than is currently available. It has been proved in |SZ] that the mul- 
tiplicity one conjecture of Gross-Prasad is true for U{n — l)-invariant 
linear forms on the smooth Frechet completions of moderate growth 
of discrete series representations of U{n) x U{n — 1), where here for 
m = n,n — 1, U{m) denotes any inner form of the compact unitary 
group U{m). In order to define Gross-Prasad periods, one needs mul- 
tiplicity one for invariant linear forms on the corresponding Harish- 
Chandra modules; in other words, one needs to know the automatic 
continuity of such invariant linear forms. This remains an open prob- 
lem, at least when U{n — 1) is non-compact and the representation of 
U{n) is not holomorphic, so in that sense the results of the present 
paper are conditional. 

The methods of this paper are in large part the same as those of my 
article |HL] with J. S. Li. The main difference is that the earlier paper 
only considered rationality criteria based on cohomological cup prod- 
ucts. In particular, we make heavy use of Li's idea to apply Flensted- 
Jensen's formula for the leading matrix coefficients of discrete series 
representations to show weak inclusion of representations of U{n — 1) 
in restrictions from U{n). We also use the method of Burger-Sarnak, or 
the more elementary methods of §7 of |H90j . to deduce non- vanishing 
of global pairings from local weak inclusion. 

We note that the proof of the full Gross-Prasad conjecture for uni- 
tary groups over archimedean fields would allow us to derive the main 
theorem without reference to the Flensted- Jensen formula. Indeed, the 
full Gross-Prasad conjecture determines the discrete series representa- 
tions of (an inner form of) U{n — 1) that are weakly contained in any 
given discrete series representation vr of (an inner form of) U{n). In 
most cases, this weak containment has nothing to do with the leading 
matrix coefficient of it. The paper is organized in a progression from 
most general and conditional results - much of the argument is valid 
for general Shimura varieties - to the most specific and unconditional. 

I thank Dipendra Prasad for asking the question answered in this 
paper and Jeff Adams for directing me to the article |SR] of Salamanca- 
Riba. I also thank the editors of this volume for inviting me to con- 
tribute. 

1. Restrictions of discrete series representations and 

global consequences 

Let G be a connected reductive algebraic group over Q, G' C G a 
reductive subgroup. Let Zq and Zq' denote their respective centers. 
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Ag and Ac the maximal R-split tori in Zq and Zq'- We assume in 
what follows that Aq is of finite index in Ac/. By a discrete series 
representation of G(]R) we will mean an irreducible Hilbert space rep- 
resentation TT of G{M.) such that, for some character ^ of vr (g)^ is 
trivial on an open subgroup A^ C Ag(M), and such that vr ® ^ defines 
a square-integrable representation of G{M.)/A^. 

Suppose TT and vr' are discrete series representations of G(M.) and 
G'(M), respectively, and assume that their central characters coincide 
on an open subgroup A^ C ^^(R). 

Hypothesis 1.1. The representation it' is weakly contained in the re- 
striction of TT to G'(M.); in other words, there is a non-trivial Hilbert 
space pairing 

TT (g) Tt''^ C 

invariant under G"(M)/y4''. 

Suppose TT occurs in the L^-automorphic spectrum of G; in other 
words, there is an equivariant homomorphism A : vr — )■ C°°(r\G'(]R)), 
for some congruence subgroup F C G(Q), such that, for some character 
^ of G{M.) trivial on F and for an open subgroup A^ C ^(^(IR) as above 
A(7r) ® ^ C L2(F • A°\G(R)). Theorem 1.1(a) of [BS] then asserts that 
tt' is weakly contained in the automorphic spectrum of G': in other 
words 

Theorem 1.2. (Burger-Sarnak) Assume the pair {tt,tt') satisfies hy- 
pothesis li.il For an appropriate character C,' o/G"(M), vr' ® ^' is the 
limit, in the Fell topology of unitary representations o/G'(R)/A°, of L"^ 
automorphic representations ofV ■ A^\G'{E.) as F' varies over congru- 
ence subgroups ofG'{W). 

If a is a Hecke character of Zq, we let L^^^p^^(G'(Q)\G'(A)) denote 
the space of complex- valued functions / on G'(Q)\G(A) with central 
character a such that, for an appropriate Hecke character ^ of G{A), 
/ ■ ^ is trivial on an open subgroup Aq C Ag(R) and defines a square 
integrable function on G{Q) ■ Aq\G{A). By a cuspidal automorphic 
representation of G we will mean alternatively an irreducible Hilbert 
space component of Ll^gp^^{G{Q)\G{A)) for some a or the correspond- 
ing {Lie{G)c, Kac) x G'(Aj))-module, for some maximal compact sub- 
group C G'(R). The first part of the following corollary follows by 
standard arguments. The second part is then obvious, and the third 
part is a consequence of the well-known fact that integrable discrete se- 
ries representations are isolated in the Fell topology (or of the Poincare 
series argument in |H90] . §7). Note that tt and tt' now denote global 
automorphic representations. 
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Corollary 1.3. Let n be a cuspidal automorphic representation of G 
whose archimedean component n^o is a discrete series representation of 
G(M). Let vr^ by a discrete series representation o/G"(]R), and assume 
the pair (vTocVT^) satisfies hypothesis 

(a) For an appropriate Hecke character C,' ofG'{A), tt'^^^'^ is the 
limit, in the Fell topology of unitary representations ofG'{M.)/A^, 
of archimedean components of automorphic representations 



(b) In particular, if vr^ is isolated in the automorphic spectrum of 
G' , then there is an L"^ -automorphic representation a of G' with 
archimedean component ti'^ and automorphic forms f E t: and 
f G vr' such that 



JG'{Q)-A0\G'{A) 

(c) If tt'^ belongs to the integrable discrete series, then there exists 
a as in (b). 



In what follows, G is a connected reductive group over Q such that 
G(M) has a discrete series. Our goal in this section is to state a simple 
algebraic condition on vr as in the statement of Hypothesis 11.11 that 
guarantees that the condition of the hypothesis is verified. We are 
not striving for maximum generality, and the condition is certainly 
unnecessarily strong. 

Choose maximal compact connected subgroups C G(R) and 



K'^ C G'(R), with K'^ C K*^. Define = K*^ ■ ZaiM), K'^ = 



K'^ ■ ZG"(ffi). Let TT be a discrete series representation of G'(M). It 
has a minimal, or Blattner /Too-type r, an irreducible representation r 
of Koo with the property that r occurs in vr with multiplicity 1, and 
is minimal in the sense that its highest weight is shortest among all 
-f^oo-types occurring in tt. 

In the next section we recall the formula for the highest weight of r 
in terms of the Harish-Chandra parameter of vr. On the other hand, 
given an irreducible representation r of Kao-, the article [SRj provides a 
necessary and sufficient criterion for r to be the minimal i^'oo-type of a 
discrete series representation (and more generally, of a cohomologically 
induced representation of the form A^{X)). Choose a maximal torus 
H C Koo- Since (j'(R) has discrete series, H is also a maximal torus of 
G. We use lower case gothic letters to denote (complexified real) Lie 
algebras; thus t = Lie{Koo)c, = Lie{H{C)), and so on. Denote by 



ofG'iA)/A^. 




2. Restrictions of minimal types 
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A{i, t) (resp. A(g, t))the set of roots of t in t (resp. g) and let (•, •) be 
the pairing on t* induced by any non-degenerate G'(]R)-invariant pairing 
on Q. 

Let fj, be the highest weight of an irreducible representation r of K^o, 
relative to a system of positive roots of t. Define 

(2.1) /x' = /i+ Yl 

agA(t,i),(a,^t)>0 

The ^-stable parabolic defined by fi' is the parabolic subalgebra q(/u') 
spanned by t and the root spaces Qa for > 0. The following 

observation is clear 

Observation 2.2. For fi sufficiently regular, c{{fi') is a Borel subalgebra 
ofQ- 

Denote by u(/i') the unipotent radical of q(/i'), and let 2p(u(/i') de- 
note the sum of the roots of t occurring in u(/i'). The following propo- 
sition is a special case of a result of Salamanca-Riba. 

Proposition 2.3. (^fjSRj, Proposition 4-V) Define 

A(u(/i'),t) = {aG A(0,t) I >0}. 

Then r is the lowest Koo-type of a discrete series representation it of 
G(M.) if and only if 

(a) q(yu') is a Borel subalgebra of g; 

(b) 

(/^',a)>(2p(u(/i')),«) 
for all a e A(u(/x'),t) 
In that case, the Harish- Chandra parameter ofn is equal to fi'—2p{u{fi')). 

When q(/i') is not a Borel subalgebra, Salamanca-Riba determines 
the parameters A for which r is the lowest i^oo-type of a module 
^q(Ai')(-^)' under a condition analogous to (b). We let G" C G be a 
reductive subgroup as in the previous section, and we assume G'(M) 
has a non-trivial discrete series. In the next few propositions, a char- 
acter A of t is "sufficiently regular" if |(A,a)| > N{G) for all roots a, 
where N{G) is a positive constant, depending only on G(M), that can 
be made explicit. 

Corollary 2.4. Let t be an irreducible representation of K^^ with high- 
est weight ji. For /i sufficiently regular, r is the lowest Koo-type of a 
discrete series representation vr o/G(]R). Moreover, for fi sufficiently 
regular, the restriction of r to K'^ contains an irreducible representa- 
tion t' that is the lowest K'^-type of a discrete series representation tt' 
ofG'iR). 
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Proof. The set of subsets of A{q, t) is finite, so the term 2p(u(/i')) that 
appears in (b) of |SRj belongs to a bounded subset of the vector space 
Hom{t, M). This imphes the first part of the Corollary, and the second 
part follows by applying the first part to G', bearing in mind that 
the highest weight of r is an extreme weight of a constituent of its 
restriction to K'^. □ 

Proposition 2.5. Let tt be an irreducible discrete series representa- 
tion of G(R) with Harish- Chandra parameter X^^. Suppose A^r is suf- 
ficiently regular. Then there is a discrete series representation tt' of 
G'(R) weakly contained in the restriction of it to G'(R). Moreover 
(possibly under a more stringent regularity condition) we can assume 
tt' to be in the integrable discrete series. 

Proof. The proof is based on the argument of Li used in |L90t IHL] . Let 
tt' be the discrete series representation of (j"(M) mentioned in the state- 
ment of \2.A\ and let t' be its minimal i^'^-type. Let ipj^ be the matrix 
coefficient of tt occurring in its minimal i^T-type r, defined by formula 
(1.2.6) of |HLj . Proposition 1.2.3 of |HLj . based on the calculations in 
§4 of |L90j . asserts that tc' is weakly contained in the restriction of tt 
provided three conditions are satisfied. First, the restriction of tpj^ to 
G' has to be in L^"*"'^(if); the argument, valid for any tempered rep- 
resentation of G, is reviewed in the course of the proof of Proposition 
2.3.4 of |HLj . Next, ipj,- has to satisfy the Flensted- Jensen formula of 
[FJj : but this is true for all discrete series representations. The final 
condition is that r' be contained in the restriction to K'^ of r; this is 
our initial hypothesis. 

The final claim follows from the characterization of integrable dis- 
crete series by Hecht and Schmid [HSj : any discrete series with suffi- 
ciently regular Harish-Chandra parameter is integrable. 



We return to the global situation: G D G' are connected reductive 
groups over Q, and both G{M.) and G"(M) are assumed to have discrete 
series. Combining |2.5l with ll.3l we draw the following global conclusion. 

Proposition 2.6. Let tt be a cuspidal automorphic representation of 
G{A), with archimedean component tt^q. Suppose the Harish-Chandra 
parameter ofn^ is sufficiently regular. Then there is an integrable dis- 
crete series representation tt'^ of G'{W), a cuspidal automorphic rep- 
resentation tt' of G' with archimedean component tt'^ and automorphic 
forms f E TT and f G tt' such that 



□ 
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Moreover, f and f can be taken in the minimal K- (resp. K'-) types 
of the archimedean components of their respective representations . 

Proof. We are changing notation: the representation vr' here is in 
\1.3\ and we have vr^ pairing with rather than weakly contained in tToo- 
Since 11.31 only claims that cr is an L^-automorphic representation, we 
need to show that in fact n' is cuspidal. If not, then tt' is a resid- 
ual square-integrable representation. But since vr^ is tempered, this 
contradicts Wallach's theorem |Wa] that residual representations are 
non-tempered at all places. □ 

If we assume that automorphic representations with discrete series 
components at infinity are isolated in the automorphic spectrum, then 
we can relax the regularity condition in the last proposition, using the 
Burger-Sarnak theorem. 

3. Coherent cohomology of unitary group Shimura 

varieties 

Preliminary notation. Let be a totally real field, F a totally 
imaginary quadratic extension of F~^. The quadratic Hecke character 
of F~^ attached to the extension F is denoted ep- Let W be an n- 
dimensional F-vector space, endowed with a non-degenerate hermitian 
form < •, • >wy relative to the extension F/F~^. We let S"*", resp. Tip, 
denote the set of complex embeddings of F~^, resp. F, and choose a CM 
type S C S^, i.e. a subset which upon restriction to is identified 
with S+. Complex conjugation in Gal{F / F^) is denoted c. 

The hermitian pairing (•, •)w defines an involution c on the algebra 
End{W) via 

(0.1) {a{v),v')w = {v,a''{v'))w, 

and this involution extends to End{W ®q R) for any Q-algebra R. 
We define the algebraic group U{W) = U{W, (•, •)w) (resp GU(W) = 
GU(W, (•,«)vi/)) over (resp. Q) such that, for any F^-algebra R 
(resp. Q-algebra S) 

U{Wm = {ge GL{W R)\g- c{9) = 1}; 

GU{W){S) = {ge GL{W ® R) \ g ■ c{g) = u{g) for some u{g) G i?^}; 

The quadratic Hecke character of corresponding to the exten- 
sion F/F~^ is denoted 

VF/F+ : A^+/F+'XiV^/^+A^ ^ ± 1. 
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If is a 1-dimensional hermitian or skew-hermit ian space then we 
write U{W) = U{1). The F+-rational points of U{1) are given by the 
group of elements x E with Np/p+{x) = 1. 

Representations of groups over archimedean local fields are assumed 
to be admissible smooth Frechet representations of moderate growth, 
in the sense of Casselman and Wallach (cf. [C]), and linear forms on 
such representations are assumed to be continuous with respect to the 
Frechet topology. The relation between such linear forms and linear 
forms on the corresponding Harish- Chandra modules is discussed in 
the appropriate place. The Harish-Chandra {Lie{G) , KoQ)-Taodu\e at- 
tached to a smooth Frechet representation 11 of the group G (and a 
choice of K^) is denoted n*^*^^. Similarly, if 11 is a representation of 
an adelic group G{A) of the form IIoo ® Hj, where 11 j is an admissi- 
ble representation of G{Af) and IIoo is an admissible smooth Frechet 
representation (of moderate growth) of the archimedean part of G{A), 
then we write n(°) = (g) Uf. 

3.1. Multiplicity one and Gross-Prasad periods. Let W be an n- 

dimensional hermitian space as above, and write W as the orthogonal 
direct sum W = W © Wq over F, where dimiy' = n — 1. Let G = 
GU{W) as above, and let G' = GU{W') x GU{Wo) n G; it is the 
subgroup of {g',go) G GU{W') x GU{Wo) with u{g') = u{go). In this 
section we apply the results of §2 to the inclusion G' C G. The new 
feature in this special case is that when vr^vr' is a cuspidal automorphic 
representation of G x G', the space of global pairings vr (g) tt' — C is of 
dimension at most 1. 

Let X and X' be G(]R) and G"(]R)-homogeneous hermitian symmetric 
domains, respectively, so that (G", X') (G, X) is a morphism of 
Shimura data. We choose maximal connected compact (mod center) 
subgroups K C G(R) and K' C G'(R) n K as in §2. Then K = 
StabG{R){x) for some x G X' C X , and we assume x to be a CM point 
(the fixed point in X' of a Q-rational torus). We are thus able to 
apply the theory of [H12j . Suppose vr and vr' are cuspidal automorphic 
representations of G and G' respectively. As in §4.3 of |H12j . there are 
number fields E{ti) and E{ti') (depending on x) over which ti^^^ and 
7r''(°) have rational structures, denoted 7r^(^^ and Tr'^^^iy 

As in |H12j . we let if C G and H' C G' denote the unitary sub- 
groups, f)' = Lie{H'). We define 

Lo(7r,7r') = iiom((,,,x')xH'(A,)(^^°^ ® ^''^°^C). 

We assume the automatic continuity hypothesis (Conjecture 4.3.1 of 
[H12] ): every element of Lo(vr,7r') has a natural continuous extension 
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to the admissible smooth Frechet representations of moderate growth 
associated to tt and tt'. It then follows from [SZJ that Lo(7r,7r') is a 
space of dimension at most 1 (cf. |H12] . Corollary 4.3.2). We also 
consider the variant 

Lr*(7r,7r') = L^°\'Ky) = ffom(f,,,;,,)x^,,(A,)(vr'"°*®7r''™''*, E(7r)®E(7r')®C) 

where 7r™°* = iif^^^ ®q C, tt'-™"* = 7r^^°j, C. Corollary 4.3.8 of |HT2] 
asserts that L[J^°*(7r, tt') is either or a free E{ti) ® E{tt') ^ C-module 
of rank 1 (there is a misprint in loc. cit.), and it follows as in the 
beginning of §4.4 that 

Proposition 3.1. Assume L^"^ , n') ^ 0. Then the free rank 1 
Eln) (g) E{n') (g) C-module L™°*(7r, tt') has a generator I'^°\7r, tt') such 
that 

I-\n, ® vr^^j,)) = E{^) ® E{n'). 

(This corrects another misprint in loc. cit..) 

On the other hand, the integral (cf. pT2] . (4.2.3)) 

(3.2) f®f^! f{h')f{h')dh' 

J H'(F)\H'{A) 

defines elements of 

as a varies over Hom{E{n)^E{'n-'), C), and thus an element P°-"'{tt, tt') G 
L™°*(7r,7r') (mislabelled /™°* in (4.4.1) of [HT2] l The Gross-Prasad pe- 
riod invariant is the constant 

P(7r,7r') G {E{tt)®E{tt')®C), 

well-defined up to a factor in (E^tt) E^tt'))^ , such that 

J™"(7r, tt') = P{n, vr') ■ /'""'(vr, vr'). 

Again, the definition of the Gross-Prasad period is conditional on the 
automatic continuity hypothesis. 

In loc. cit. a conjecture is assumed that implies that either P(7r, tt') = 
or P{tt, tt') G {E{it) (g) E{'k') ® C) ^ . In order to avoid referring to this 
conjecture - and to avoid creating new misprints - we henceforward 
fix a G Hom{E{n) ® E{tx'), C) of E{'n) ■ E{'k') and consider vrJ^^J^) and 

'^'e(1') ^ subspaces of the respective complex automorphic representa- 
tions. Then we let Pa(7r, vr') G C be the projection of P{tt,tt') on the 
a-component; Pa(7r, vr') = if and only if fH'(F)\H'{A) fi^')f'i^')^^' — 
for all f e-K, f e it'. 

We reformulate 12.61 in terms of the Gross-Prasad periods. 
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Proposition 3.3. Let tt be a cuspidal automorphic representation of 
G{A), with archimedean component tToo in the discrete series. Suppose 
the Harish- Chandra parameter of ti^o is sufficiently regular. Then there 
is an integrable discrete series representation vr^ o/G'(M), a cuspidal 
automorphic representation ir' of G' with archimedean component vr^ 
such that /^""(vr, vr') 7^ 0. Moreover, we can find / G vr and f G tt' in 
the minimal K- (resp. K'-) types of the archimedean components of 
their respective representations such that /'^""(vr, 'n''){f, /') 7^ 0. 

Corollary 3.4. Let vr and tt' be as in the previous proposition and a 
as in the discussion above. Then Pa{TT , n')^^ ■ P"-"- {n , tt') is a non-zero 
a{E{TT) (g) E{tt'))- bilinear pairing 

that does not vanish on the tensor product of the minimal types at 
infinity. 

Let TT^'' and tt'-^^^ be the Q-subspaces generated by t^e{tt) ^^'-^ ''^^{'1') 
in the respective complex automorphic representations. 

Theorem 3.5. Assume the automatic continuity hypothesis (Conjec- 
ture 4-3.1 of |H12j ) Let tt be a cuspidal automorphic representation of 
G{A), with archimedean component tt^ in the discrete series and min- 
imal K-type T. Suppose the Harish- Chandra parameter of tt 00 is suffi- 
ciently regular. Let f E tt be an automorphic form whose archimedean 
component lies in r; in particular, f E tt^^'^ . Let n'^ be a discrete se- 
ries representation of G'{R) that satisfies the conditions of \3.3[ Then 
f G TT^^ if and only if, for every 7 G G{Af), every automorphic repre- 
sentation tt' of G' of infinity type tt'^, and every f G tt^, 

(3.6) A,,^,(/) :=P,(7r,7r')-^-/^'^"(7r,7r')(r,/0 gQ 

where f^{h) = fi^h^y). 

Proof. The proof is exactly analogous to that of Theorem 7.6 of [H90j . 
We temporarily adopt the notation of that proof: thus tt contributes 
to the coherent cohomology space -ff'^([V]) for some irreducible auto- 
morphic vector bundle [V] on the Shimura variety attached to G. The 
space H'^ of |H90] is elsewhere called interior cohomology; in |H12] it 
is denoted H^. The regularity hypothesis implies that the vrj-isotypic 
component of ff'^([V]) consists entirely of the classes of cusp forms (cf. 
|H12] . 3.3.2); in particular, hypothesis (d) of |H90] 7.6. Hypotheses (a) 
and (c) hold for similar reasons: the point is that we can take t to be 
a Cartan subalgebra of Lie{G') as well as Lie{G), and the walls of a 
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Weyl chamber for G' form a subset of the walls of a Weyl chamber for 
G, so if an infinitesimal character in t* is far from the walls for G it is 
a fortiori far from the walls for G' . Condition (b) of |H90] . 7.6, which 
identifies J'^"" as a cup product in coherent cohomology, does not hold; 

instead we have the condition that dim ifom((,/ x')(^oo'' ® tt'qq^K C) = 1, 
which requires automatic continuity. 

Now the rational structure vr^'' is invariant under the action of G{Af), 

so it is obvious that if / G vr^^ then (13.61) is satisfied for all vr' and all 
/' G vr^. So it suffices to show that the linear forms A^j/ span the space 
of linear forms on the (count able- dimensional) subspace r (g) ttj of vr. 
This is the content of Theorem 7.4 of [H90j : the condition 7^ p{v) in 
the statement of that theorem is implied by the assertion concerning 
minimal types in 13.31 

□ 

Remark 3.7. Again, if we know a priori that tt^ is isolated in the 
automorphic spectrum of G', then we can apply |BS] rather than the 
arguments in [H90j . 

Remark 3.8. By using the more precise Gross-Prasad periods defined 
in [H12j . we can obtain a criterion for / to be rational over the field 
E{7r), and not just over Q. The condition is that Gal{Q/E{7r)) acts on 
the quantities X-yj'if) by permuting every appearance of /, /', vr, vr', 
and a, including in the period invariant appearing on the right-hand 
side of flMjl . 

Theorem 13 . 5 1 characterizes Q-rational forms on G of (sufficiently reg- 
ular) coherent cohomology type by their pairings with Q-rational forms 
on G' of coherent cohomology type, but of course one needs to be able 
to characterize the latter. However, it was noted during the proof that 
the regularity hypothesis for tToo implied a regularity hypothesis for n'^. 
Indeed, it follows from the classical branching laws for the restriction 
of a representation of (compact) U{m) to U{m — 1) that conditions 
(a) and (b) of 12.31 for the minimal K-tjpe t imply the corresponding 
conditions for at least one irreducible constituent r' of its restriction to 
K'. Thus by induction on n we can assume that the Q-rational /' used 
to define the A^j/ have already been identified. This gives a complete 
answer to the question of Piatetski-Shapiro mentioned in the introduc- 
tion in the case of Shimura varieties attached to unitary groups, at 
least under a regularity hypothesis. I don't know whether or not the 
regularity hypothesis can be relaxed. 
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Remark 3.9. The rationality criterion of Theorem 13.51 makes specific 
use of the representation tt^ that can be proved to pair non-trivially 
with TToo, thanks to the positivity property of the Flensted- Jensen min- 
imal matrix coefficients. If we know independently that (the dual of) 
TT^ is weakly contained in tToo, then the analogue of Theorem [33] re- 
mains valid. The pairings whose existence is predicted by |GGP] do not 
restrict in most cases to non-trivial pairings of minimal types; neverthe- 
less, the Gross-Prasad conjecture for real unitary groups implies that 
each non-trivial pairing gives a new rationality criterion for coherent 
cohomology. 

4. GROSS-PRASAD PERIODS AS CUP PRODUCTS 

The novel feature of this article is the introduction of a rationality 
criterion, valid for automorphic representations of sufficiently regular 
discrete series type, that, unlike the criterion of |H90l IHL] . is not de- 
rived from a cup product in coherent cohomology. In this final section 
we characterize cup-product pairings among all pairings given by inte- 
grals over H'{A), and show that certain tToo never admit cup product 
pairings with coherent cohomology classes on the Shimura variety at- 
tached to H'. Thus the rationality criterion of the present paper is 
substantially more general than that of |H90] . 

The cup-product pairings are of independent interest, since they can 
be used to study period relations. This will be explained in forthcoming 
work. 

4.1. Parameters. In this section we let H denote the real Lie group 

U{r, s), the unitary group of the standard hermitian form = 

The maximal compact subgroup K = U{r) x U{s), the subgroup that 
respects the decomposition of C" into positive-definite and negative- 
definite subspaces as indicated by the form of Ir^s- For a maximal 
torus T we take the group of diagonal matrices in H; this is also a 
maximal torus of K. An irreducible finite-dimensional representation 
(cr, V) of H is determined by its highest weight a„ relative to T and the 
upper-triangular Borel algebra b of () = Lie{H)c', cLa is written in the 
usual way as a non-increasing n-tuple of integers (ai > 02 > ■ ■ ■ > a^). 

Let p = (^^, . . . , ^Y^) be the half-sum of positive roots for 
b. We let p = p + ^(1, !,...,!) = (n - l,n - 2,..., 1,0). The 
infinitesimal character of the representation {a, V) above is the element 
Ag- = Oo- + p of t*; the element = a„ + p = {ai + n — i,i = 1, . . . ,n) 
will be called the Hodge parameter, for reasons that will soon be clear. 
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We consider the L-packet 11^ = H^''^ of discrete series representations 
TT of if such that H^{i, tt (g) V^) ^ for some % (necessarily % — ts — 

dimif/fC). Then 11^ contains members, indexed by permutations 

of the entries of \„ or modulo (subsequent) permutations that 
preserve the first r entries. Write 

= (Ai > A2 > ■ ■ ■ > A„); A^ = (Ai > A2 > ■ ■ ■ > An) 

It's more convenient to index Ho- by (r, s)-shuffles of the Aj, namely 
(r, s)-tuples of the form A = (oi > • • • > a^; 61 > • • • > 65) which is a 
permutation of the entries of A(j. If tt e corresponds to A, then A 
is the Harish- Chandra parameter of tt, and we write tt — tt^. We also 
define A = A + ^(1, 1, . . . , 1). 

Let A'^'^'^ be the set of positive non-compact roots - roots of the 
form 

ai^r+j ■ (Cl, C2, . . . , C„) H- Q - Cr+j, I < i < T.l < j < S. 

These are by definition the roots of T acting on p~ C (), the Lie alge- 
bra of matrices of the form , which is also isomorphic to the 

antiholomorphic tangent space of the hermitian symmetric space H/ K 
at the fixed point of K. In other words, the choice of b, and therefore 
p, is consistent with a specific choice of complex structure. The degree 
q{\) of A is the number of pairs such that > bj, or equivalently 
to the cardinality of 

A"^'+(A) = {a e A"^'+ I {a, A) > 0. 

This is related to the usual notion of length by the formula q{X) — 
rs — i{w), where w is the permutation that takes A^. to A. There are 

two extreme elements of ITo-: the antiholomorphic representation, with 
Harish- Chandra parameter (and degree rs), and the holomorphic 
representation, corresponding to the longest shuffle, equivalently to the 
Harish- Chandra parameter (A5+1 > • • • > A„; Ai > • • • > As) (with 
degree 0). 

The integer g(A) is the degree of coherent cohomology to which A con- 
tributes. More precisely, to any A we associate its coherent parameter 
A — X — p, viewed as the highest weight of an irreducible representation 
Wa of K. Let q = p- e Lie{K). Then 

(4.1) dimHi^^\q,K;7rx®WX) 

and all other H'^{q,K;7ix^W) vanish as W runs over irreducible rep- 
resentations of K. 
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The space in|Ois spanned by an element of (A'''(-^)(p Y^nx^W)^)^, 
or equivalently by a homomorphism 

(4.2) hx G HoTTiKiA'^^Hp-) ® WA,nx). 

The image of hx is an irreducible i^'-type tx, the minimal (or Blattner) 
K-tjpe, and its highest weight, also denoted r, is the Blattner param- 
eter of tta (or of A). The formula for r is given in terms of the chamber 
C(A) for which A is positive 

(4.3) r = A+ a. 

Q,gA"<='+(A) 

For each a G A"^'"*", choose a non-zero basis vector in the corre- 
sponding root space. 

Lemma 4.4. The homomorphism hx is non-zero on the vector 

where wa is a basis vector of the highest weight subspace of Wa, and 
h{vx) generates the highest weight space ofr. 

Proof. This is presumably well-known but I include the one-line proof: 
if we take a positive root system for K compatible with the chamber 
C(A), then vx, which is a weight vector for the weight is an extreme 
vector in this chamber in the tensor product A^*^'*'^(p~) (g) Wa and in 
particular has multiplicity one. Since the image of h^- contains a vector 
of weight 14.31 it can't vanish on vx- 

□ 

4.2. Rationality. Now we return to the notation of §3, with G = 
GU{W), G' = G{U{W') X U{Wo)). The elements of the CM type E 
are denoted v, and for each v the space Wy = W ^f,v 'C is a complex 
hermitian space with signature (r^,, s^). We assume that W^ has signa- 
ture {vy — l,Sy) for each v; although the discussion can be modified if 
this is not the case, we can always choose W' with this property, and 
it simplifies the discussion. 

A discrete series representation vr of G{M.) restricts to a sum of dis- 
crete series representations of H{R) = Yl^ U{ry, Sy) := Y[y Hy. In fact, 
there is no ambiguity unless r„ = Sy for some f ; each such place con- 
tributes two summands, and we choose one of the two - which we still 
call vr - and replace Hy at such a point by its (real) identity compo- 
nent . We thus obtain a S-tuple of Harish-Chandra parameters 
(A„,f G S). Let Ky = U{ry,0) x U{0,Sy) C U{ry,Sy) be a maximal 
compact subgroup, as above, and let K = Y[y Ky. As above, the sig- 
nature determines a holomorphic structure on the the symmetric space 
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X{G)^ = H{R)/K - the product over v of the spaces denoted above 
is the space of anti-holomorphic tangent vectors at the fixed point of 
K in the symmetric space - and therefore we can assign to each v a 
reference parameter 

A(T,t) = {^i,v > ■ ■ ■ > A„^^) 

which is the infinitesimal character of an irreducible finite-dimensional 
representation (cXt,, K) of H^. This is a permutation of A^, which we 
write as above 

At, = (ai^t, > ■ ■ • > flr,;,?,; bi^y > ■ ■ > hs^^v)- 

Let A^, = At, — be the f-component of the coherent parameter of 
vr, A = (At,,f G S). It defines an automorphic vector bundle \W^\ on 
the Shimura variety Sh{G, X{G)), where X{G) is the (disconnected) 
hermitian symmetric space attached to G{M) and X{G)^ C X{G) is 
a fixed if(]R)-invariant component (also invariant under the identity 
component G(M)'' C G(M). As in the proof of 13.51 we follow [H90] and 
denote by i/*([WA] the interior cohomology of [PVa]. 

Let H' = U{W') X UiWo), K' = ]\^K'^, K', = n H'{R) = 
U{ry — 1, 0) X f/ (1, 0) X f/ (0, Si,); we drop the O's in what follows. Write 
A^ = (ai^t, > ■ ■ ■ > ar„,t,; Pi,v > ■ ■ ■ > Ps^,v)] this is a dominant integral 
weight of a finite-dimensional representation of K^. It follows 
from the classical branching formula that the restriction of to K'^ 
contains the representation with highest weight 

where the semicolons separate the weights for U^r^) x f/(l) x f/(s„). 
Let A' = (A^) be the corresponding highest weight for K', and let 
A' = A' + p', where p' is by analogy the half-sum of the positive roots of 
H' relative to T for the chamber containing the chosen positive chamber 
for {H,T). With the above notation, we have 

(4.5) ^ 111 
A^ = (ai^t, — ->■■■> ar„-i,u -; Oir^y, bi^v 2 ^ ' ' ' ^ ~'~ 2''' 

Hypothesis 4.6. The highest weight aa,v = ^(t,v — P of (cr^,K) is a 
regular character of T for each v. Equivalently, for each v, Aj,„ — 
Ai+i,„ > 2, i = 1, . . . ,n - 1. 

Lemma 4.7. Under Hypothesis \4.6[ A' is regular for the root system of 
{H',T), and belongs to {^^ + Z)'^~^ . It is therefore the Harish- Chandra 
parameter for a unique discrete series representation ny of H'{M.). 
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This is clear from the form of 14.51 under the regularity hypothesis, 
none of the coincide with any of the bj^y + i. 

We now consider the following hypothesis: 

Hypothesis 4.8. For each v, the parameter ar^^y equals Xn,v, the small- 
est of the Xi^y. 

Of the Yly f ^ ] elements ttx C H^, the set of parameters satisfying 



i-v \ r 



IS 



n — \ 

Sv I T-r 



n\ y ^ 

Sy) 

of the total. 

Let A''"'^'"'" be the set of positive non-compact roots for {H',T), de- 
fined as in the previous section. We can view A'*"'^'"*' as the set of 
ai,r^+j G A"'^'+ with 1 < i < r^, — 1; 1 < j < s„ for f G S. Hypothesis 
14.81 is equivalent to the hypothesis that 

(4.9) A''""'+(A') = A""'+(A). 

We define X{G')+ C X{G'), q', and the Shimura variety Sh{G', X{G')), 
as in the previous paragraph; we also define A' = X' — p', the rep- 
resentation Wa' of K', and the automorphic vector bundle [Wai] on 
Sh{G',X{G')). The inclusion ofthe Shimura data (G',X(G")) C {G,X{G)) 
defines an inclusion of Shimura varieties, and by functoriality a restric- 
tion map of (9-cohomology 

(4.10) H'^^^\q,K-7ix(»WX) -> H'i^^^\q',K';ny®W;^,). 
It follows immediately from Lemma 14.41 and 14.91 that 

Proposition 4.11. Assume Hypothesis \4-6[ 

(a) // A satisfies Hypothesis \4-S[ then the map (J^TU^ is an isomor- 
phism of 1-dimensional spaces. 

(b) If X does not satisfy then Ili4.10[ ) is the zero map. 

For global automorphic representations vr and vr' whose archimedean 
components satisfy Hypothesis l4.8l the canonical period integral /™"'(7r, it') 
can be identified with cup products in coherent cohomology, by the 
formalism of [H901 IHLj . The following theorem is an immediate conse- 
quence of Lemma 7.5.5 of |H90] and the definitions: 

Theorem 4.12. Suppose X satisfies Hypotheses \4.6\ and \4.S\ Let it and 
Tc' be cuspidal automorphic representations of G and G', respectively, 
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with archimedean components tta and Tty . Then the Gross-Prasad pe- 
riod invariants Po,{7!' , 7!'') are algebraic numbers. 

This can be refined to give vector-valued Gross-Prasad invariants, 
depending on complex embeddings of the coefficient fields, whose en- 
tries belong to specific number fields. Combining Theorem 14.121 with 
ergodic considerations, along the lines of II. 3^ we obtain examples of 
the rationality criteria provided by Theorem 7.6 and Corollary 7.7.1 of 
|H90] : the point is that (a) of Proposition 14.111 implies hypothesis (b) 
of Theorem 7.6 of loc. cit. Theorem 14.121 is also of interest in proving 
period relations, and will be considered at length in a future paper. 

On the other hand, (b) of Proposition 14.111 together with Theorem 
13.51 gives explicit examples of rationality criteria that have nothing to 
do with coherent cohomology. It is plausible that when vr does not 
satisfy 14.81 then there is no vr' pairing non-trivially with vr for which 
the PaiiTjTr') are algebraic. On the other hand, it is possible that vr"^ 
satisfies 14.81 even though vr does not. Since a rationality criterion for 
TT^ easily gives rise to one for vr (using cup products on H rather than 
on H'), it is natural to wonder whether for all vr satisfying 14.61 either 
vr or tt"^ also satisfies 14.81 This seems unlikely: in the notation of 14. 8[ 
suppose ar^^y = Xn,v for some but not all v. The above reasoning then 
suggests that, except possibly for some degenerate cases, neither vr nor 
TT^ satisfies 

References 

[BS] M. Burger and P. Sarnak, Ranianujan Duals II, Invent. Math., 106 (1991), 
1-11. 

[C] W. Casselman, Canonical extensions of Harish- Chandra modules to represen- 
tations of G, Can. J. Math., 41, (1989) 385-438. 

[CI] L. Clozel, Motifs et formes automorphes, in Automorphic Forms, Shimura 
Varieties, and L-functions, New York: Academic Press (1990), Vol. 1, 77-159. 

[D] P. Deligne, Valeurs de fonctions L et periodes d'integrales, Proc. Symp. Pure 
Math., XXXIII, part 2 (1979), 313-346. 

[FJ] M. Flensted- Jensen, Discrete series for semisimple symmetric spaces, Ann. of 
Math., Ill (1980), 253-311. 

[GGP] W. T. Can, B. Gross, and D. Prasad, Symplectic local root numbers, cen- 
tral critical L- values, and restriction problems in the representation theory of 
classical groups, Asterisque, 346 (2012). 

[II90] M. Harris, Automorphic forms of 9-cohomology type as coherent cohomology 
classes, J. Diff. Geom., 32, (1990) 1-63. 



TESTING RATIONALITY OF COHERENT COHOMOLOGY 



19 



[H12] M. Harris, Beilinson-Bernstein localization over Q and periods of automor- 
phic forms. Int. Math. Res. Not. (2012) doi: 10.1093/imrn/rnslOl 

[HL] M. Harris and J.-S. Li, A Lefschetz property for subvarieties of Shimura vari- 
eties. J. Algebraic Geom., 7 (1998) 77122. 

[HLS] M. Harris, J.-S. Li, B.-Y. Sun, Thcta correspondences for close unitary 
groups, in J. Cogdell, J. Funke, M. Rapoport, T. Yang, eds., Arithmetie Ge- 
ometry and Automorphic Forms, Beijing: International Press (2011), 265-308. 

[HS] H. Hecht, W. Schmid, On integrable representations of a semisimple Lie group. 
Math. Ann., 220 (1976) 147-149. 

[L90] J.-S. Li, Theta liftings for unitary representations with non-zero cohomology, 
Duke Math. J., 61 (1990) 913-937. 

[SR] S. Salamanca-Riba, On the unitary dual of real reductive groups and the 
Aq(A) modules: the strongly regular case, Duke Math. J., 96 (1998) 521 546. 

[SZ] B. Sun and C.-B. Zhu, Multiplicity one theorems: the archimedean case. 

[Wa] N. Wallach, On the constant term of a square integrable automorphic form, in 
Operator algebras and group representations. Vol. II (Neptun, 1980), 227237, 
Monogr. Stud. Math., 18, Boston: Pitman (1984). 



Institut de Mathematiques de Jussieu, U.M.R. 7586 du CNRS; UFR 
DE Mathematiques, Universite Paris-Diderot Paris 7 



